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DETAILED ACTION 



Drawings 

1 . The corrected or substitute drawings were received on 3/7/2003. These drawings are 
acceptable. 

Claim Rejections - 35 USC § 102 

2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

3. Claims 21-23, 26-28, 45, and 46 are rejected under 35 U.S.C. 102(b) as being anticipated 
by Brodsky (US 5,809,471). 

As to claim 21, the claimed receiving content comprising a video image; receiving a 
keyword embedded in the video image; searching a network for information relating to the 
keyword; and receiving the information relating to the keyword. The Brodsky reference 
discloses retrieval of additional information not found in interactive TV or telephony signal by 
application using dynamically extracted vocabulary where the program may be received via any 
transmission means including radio, TV, and telephone. The apparatus includes a recognition 
system which can recognize the user's request and cause its satisfaction to be provided. This is 
accomplished by creating a dynamically changing dictionary of items or keywords extracted 
from the most recently received program portion. The user's request is matched against this 
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continually changing dictionary. Upon recognition of the request, a search is initiated to access, 
import and deliver to the user the information required to satisfy the request (Brodsky 3:56-66). 
The Brodsky reference also discloses the context extractor 102 has a data processor to extract 
words and/or items from an audio, video or telephony signal (Brodsky 4:40-42) where embedded 
keywords are inherent in order for extraction to occur. The Brodsky reference also discloses a 
high priority may be assigned to words identifying the program itself. Some of this information 
may be captured directly from the visual or audio part of the signal. It may also be captured 
from imbedded parts of the signal, such as closed caption text transmitted in video vertical 
blanking intervals (Brodsky 5:53-59). The Brodsky reference also discloses in another 
embodiment, the apparatus provides the needs for the following scenario. While watching a 
movie a viewer hears an actor refer to a character in the plot. A service provided by the movie 
program producer, broadcaster or a dial-up service provider, gives program relevant information 
upon a user's request (Brodsky 6:43-48). 

As to claim 22, the claimed displaying the video image; and displaying the keyword. The 
Brodsky reference discloses the context extractor 102 has a data processor to extract words 
and/or items from an audio, video or telephony signal. The extractor 102 samples the incoming 
signal and identifies 'words' and transmit those 'words' to the buffer for storage in the 
dictionary. The selection criteria may include matching all acceptable 'words' against an 
application-supplied listing of available topics. The signal is also passed undisturbed on to its 
normal display 108, say a TV set, radio or a telephone (Brodsky 4:40-48). The Brodsky 
reference also discloses the user interface 1 10 is a means for the user receiving the radio, TV or 
telephone signals to make requests by voice or other input request means for more video or audio 
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information about a topic recently seen or heard. The user interface gives a viewer or listener the 
ability to directly or remotely make requests to an application and to view the response. Options 
are provided for the user to view or hear the 'words' that are valid requests. Options may be 
provided for other forms of requests, say by button selection from a visual or audio menu of 
valid requests (Brodsky 5:1 1-20). The display is also used to present the menu of valid requests 
when commanded by the user. For a video display situation, the request satisfaction can use 
selective video to replace or be shown together with the normal video. This can be accomplished 
using known techniques including picture in picture and split screen. A useful option is to 
provide a means for the user to display all or part of the buffer contents to enable the user to 
choose a valid request. This can be formed in the fashion of a menu of the dictionary entries 
(Brodsky 5:26-35). 

As to claim 23, the claimed further comprising the step of displaying the information 
relating to the keyword. The Brodsky reference discloses upon recognition of the request, a 
search is initiated to access, import and deliver to the user the information required to satisfy the 
request. The Brodsky reference also discloses the viewer chooses the specific desired topic from 
the displayed menu. For example, if the user inputs the name of a character for whom additional 
information is desired, the name is compared to the menu dictionary. The recognition processor 
then compares the name against the valid names in the displayed menu. Upon making the match, 
the processor transmits the match to the database application. The desired information is 
retrieved and displayed as a subtitle type text, while the viewer is still watching the movie 
(Brodsky 6:60-67; 7:1-2). 
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As to claim 26, the claimed further comprising receiving a code included with the 
keyword; wherein the code assists in the searching of the network for information relating to the 
keyword. The Brodsky reference discloses it may be desirable to allow the setting of priorities 
for all 'items' and/or 'words'. Priorities may be set so that certain application relevant 'items' or 
'words' are kept a longer than ordinary duration, or even for the whole length of the program 
being watched (Brodsky 5:48-52). The Brodsky reference also discloses preprocessing of key 
words representing potential user requests, decreases system response time when such a request 
is subsequently made. Preprocessing is desirably performed in conjunction with the priorities of 
particular items or words. It may only search for items or words assigned with at least a 
moderately high priority level, and actually retrieve information only for items or words assigned 
with a higher priority level than that of the moderately high priority level (Brodsky 6:3-1 1). 

As to claim 27, the claimed wherein the video image and the keyword are received over a 
broadcast medium. The Brodsky reference discloses the program may be received via any 
transmission means including radio, TV and telephone (Brodsky 3:56-58). The Brodsky 
reference also discloses the context extractor 102 has a data processor to extract words and/or 
items from an audio, video or telephony signal (Brodsky 4:40-42). 

As to claim 28, the claimed further comprising the step of displaying the video image. 
The Brodsky reference discloses the signal is also passed undisturbed on to its normal display 
108, say a TV set, radio or a telephone (Brodsky 4:47-48). 

As to claim 45, the claimed receiving content comprising a video image; receiving a 
keyword associated with the video image; receiving a code relating to the keyword; and 
receiving information relating to the keyword and the code. The Brodsky reference discloses the 
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program may be received via any transmission means including radio, TV and telephone. The 
apparatus includes a recognition system which can recognize the user's request and cause its 
satisfaction to be provided. This is accomplished by creating a dynamically changing dictionary 
of items or keywords extracted from the most recently received program portion. The user's 
request is matched against this continually changing dictionary. Upon recognition of the request, 
a search is initiated to access, unport and deliver to the user the information required to satisfy 
the request (Brodsky 3:56-66), The Brodsky reference also discloses a high priority may be 
assigned to words identifying the program itself. Some of this information may be captured 
directly from the visual or audio part of the signal. It may also be captured from imbedded parts 
of the signal, such as closed caption text transmitted in video vertical blanking intervals 
(Brodsky 5:53-59). The Brodsky reference also discloses in another embodiment, the apparatus 
provides the needs for the following scenario. While watching a movie a viewer hears an actor 
refer to a character in the plot. A service provided by the movie program producer, broadcaster 
or a dial-up service provider, gives program relevant information upon a user's request (Brodsky 
6:43-48). The Brodsky reference also discloses it may be desirable to allow the setting of 
priorities for all 'items' and/or 'words'. Priorities may be set so that certain application relevant 
'items' or 'words' are kept a longer than ordinary duration, or even for the whole length of the 
program being watched (Brodsky 5:48-52). The Brodsky reference also discloses preprocessing 
of key words representing potential user requests, decreases system response time when such a 
request is subsequently made. Preprocessing is desirably performed in conjunction with the 
priorities of particular items or words. It may only search for items or words assigned with at 
least a moderately high priority level, and actually retrieve information only for items or words 
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assigned with a higher priority level than that of the moderately high priority level (Brodsky 6:3- 
11). 

As to claim 46, the claimed searching a network for information relating to the keyword 
and the code. See rejection of claim 45. 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

5. Claims 21-24, 26-35, 37-41, 43, 44, 45, and 46 rejected under 35 U.S.C. 102(e) as being 
anticipated by Portuesi (TJS 6,499,057). 

As to claim 21, the claimed receiving content comprising a video image; receiving a 
keyword embedded in the video image; searching a network for information relating to the 
keyword; and receiving the information relating to the keyword. The Portuesi reference 
discloses a system and method for activating uniform network resource locators displayed in a 
media broadcast where FIG. 5 is a block diagram of one embodiment of a system, indicated 
generally at 50, for distributing a video signal having encoded uniform network resource locators 
to a number of end points for decoding and display, where the examiner interprets a URL to be 
the equivalent of a keyword (Portuesi 8:49-52). The Portuesi reference also discloses if a URL is 
activated, URL processing unit 72 can invoke Web browser 76 to retrieve the resource located at 
the activated URL and display the resource to the user (Portuesi 9:31-34) where accessing a URL 
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requires searching at least in regard to a look up table for finding the location of the required 
resource on the Internet 

As to claim 22, the claimed displaying the video image; and displaying the keyword. The 
Portuesi reference in Figure 3 illustrates the URL, keyword, displayed on the screen with a video 
image (Portuesi, see Figure 3). 

As to claim 23, the claimed further comprising the step of displaying the information 
relating to the keyword. The Portuesi reference discloses if a URL is activated, URL processing 
unit 72 can invoke Web browser 76 to retrieve the resource located at the activated URL and 
display the resource to the user (Portuesi 9:31-34). 

As to claim 24, the claimed further comprising the step of displaying the keyword 
embedded in the video image in response to a selection of the video image. The Portuesi 
reference discloses display window 28 can include a caption 34 which provides a description of 
the area within display window 28 over which a pointing device, such as a mouse pointer, is 
positioned. For example, if the pointing device is positioned over hypertext link 22, caption 34 
can provide a name for link 32 or provide the actual URL (Portuesi 6:22-27). 

As to claim 26, the claimed further comprising receiving a code included v^th the 
keyword; wherein the code assists in the searching of the network for information relating to the 
keyword. The Portuesi reference discloses as used herein, the term "uniform network resource 
locator" denotes an identifier of a network document or other resource, formatted in accordance 
with a uniform network protocol, such that computer clients of the network for example, can 
request the document or resource from computer servers of the network by reference to the 
identifier. Internet/intranet URLS, formatted in accordance with HTTP protocol, are one 
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example of uniform network resource locator as that term is used herein (Portuesi 4:16-24), 
where the examiner views the "http://" of the URL to be a code to facilitate access of related 
information by indicating access using the HTTP protocol. 

As to claim 27, the claimed wherein the video image and the keyword are received over a 
broadcast medium. The Portuesi reference discloses FIG. 5 is a block diagram of one 
embodiment of a system, indicated generally at 50, for distributing a video signal having 
encoded uniform network resource locators to a number of end points for decoding and display. 
Specifically, the embodiment of FIG. 5 uses movie files with embedded Internet/intranet URLs. 
System 50 includes encoding system 52, a distribution network 54, and a decoding system 56. In 
general, encoding system 52 is associated with a source of the video signal such as a cable 
network or local television station, distribution network 54 is associated with distributing the 
video signal such as by the cable company or local TV station, and decoding system 56 is 
associated with end points such as personal residences (Portuesi 8:49-61). 

As to claim 28, the claimed fiirther comprising the step of displaying the video image. 
The Portuesi reference illustrates displaying the video image in FIG. 3 (Portuesi, see Figure 3). 

As to claim 29, the claimed further comprising the step of selecting the video image. The 
Portuesi reference discloses as shown, when a pointer 38 of a pointing device, such as a mouse, 
is positioned over a hot spot 40, hot spot 40 can be highlighted to indicate to the user that pointer 
38 is positioned over hot spot 40. Also, caption 34 can display a URL 42 associated with hot 
spot 40 (Portuesi 6:36-40). The Portuesi reference also discloses if a URL is activated, URL 
processing unit 72 can invoke Web browser 76 to retrieve the resource located at the activated 
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URL and display the resource to the user (Portuesi 9:32-35) where selection of a video image hot 
spot activates a URL. 

As to claim 30, the claimed displaying the keyword in response to the selecting of the 
video image. The Portuesi reference discloses display window 28 can include a caption 34 
which provides a description of the area within display wdndow 28 over which a pointing device, 
such as a mouse pointer, is positioned. For example, if the pointing device is positioned over 
hypertext link 22, caption 34 can provide a name for link 32 or provide the actual URL (Portuesi 
6:22-27). 

As to claim 31, the claimed displaying a video image; selecting at least a portion of the 
video image; and displaying a keyword associated with the portion of the video image in 
response to the selecting of the portion of the video image. The Portuesi reference illustrates the 
display of a video image in FIG. 3 (Portuesi, see Figure 3). The Portuesi reference also discloses 
display window 28 can include a caption 34 which provides a description of the area within 
display window 28 over which a pointing device, such as a mouse pointer, is positioned. For 
example, if the pointing device is positioned over hypertext link 22, caption 34 can provide a 
name for link 32 or provide the actual URL (Portuesi 6:22-37). 

As to claim 32, the claimed further comprising the step of sending the keyword over a 
network. The Portuesi reference discloses if a URL is activated, URL processing unit 72 can 
invoke Web browser 76 to retrieve the resource located at the activated URL and display the 
resource to the user (Portuesi 9:32-35). The Portuesi reference also discloses this allows a user 
to activate links and connect to resources, for example, across the public Internet or private 
intranets during playback of the time-based medium. 
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As to claim 33, the claimed further comprising the step of receiving over the network 
information relating to the keyword. The Portuesi reference discloses if a URL is activated, 
URL processing unit 72 can invoke Web browser 76 to retrieve the resource located at the 
activated URL and display the resource to the user (Portuesi 9:32-35). 

As to claim 34, the clauned further comprising the step of searching a network for 
information relating to the keyword. The Portuesi reference discloses if a URL is activated, 
URL processing unit 72 can invoke Web browser 76 to retrieve the resource located at the 
activated URL and display the resource to the user (Portuesi 9:32-35) where accessing a URL 
requires searching at least in regard to a look up table for finding the location of the required 
resource on the Internet. 

As to claim 35, the claimed wherein the keyword is embedded in the video image. The 
Portuesi reference discloses the encoded information defines a uniform network resource locator 
embedded in the video (Portuesi 3:15-16). 

As to claim 37, the claimed displaying a video image; selecting at least a portion of the 
video image; and sending over a network a keyword associated with the portion of the video 
unage in response to the selecting of the portion of the video image. The Portuesi reference 
discloses a video image with an associated hot spot in FIG. 4 (Portuesi, see Figure 4). The 
Portuesi reference also discloses as shown, when a pointer 38 of a pointing device, such as a 
mouse, is positioned over a hot spot 40, hot spot 40 can be highlighted to indicate to the user that 
pointer 38 is positioned over hot spot 40. Also, caption 34 can display a URL 42 associated with 
hot spot 40. Thus, active embedded URLs in a video playback can be displayed as hot spots in 
addition to being displayed as hypertext links. In essence, the URL track can define an image 
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map on top of a video display and provides active URLs to the user (Portuesi 6:36-44). The 
Portuesi reference also discloses if a URL is activated, URL processing unit 72 can invoke Web 
browser 76 to retrieve the resource located at the activated URL and display the resource to the 
user (Portuesi 9:32-35) where an URL is activated my selecting a hot spot portion of a video 
image and URL (keyword) is sent over a network to retrieve a designated resource. 

As to claim 38, the claimed wherein the keyword includes a code. The Portuesi reference 
discloses as used herein, the term "uniform network resource locator" denotes an identifier of a 
network document or other resource, formatted in accordance with a uniform network protocol, 
such that computer clients of the network for example, can request the document or resource 
from computer servers of the network by reference to the identifier. Internet/intranet URLS, 
formatted in accordance with HTTP protocol, are one example of uniform network resource 
locator as that term is used herein (Portuesi 4:16-24), where the examiner views the "http://" of 
the URL to be a code to facilitate access of related information by indicating access using the 
HTTP protocol. 

As to claim 39, the claimed further comprising the step of displaying the keyword 
associated with the portion of the video image in response to the selecting of the portion of the 
video image. The Portuesi reference discloses display window 28 can include a caption 34 
which provides a description of the area within display window 28 over which a pointing device, 
such as a mouse pointer, is positioned. For example, if the pointing device is positioned over 
hypertext link 22, caption 34 can provide a name for link 32 or provide the actual URL (Portuesi 
6:22-27). 
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As to claim 40, the claimed wherein the keyword is embedded in the video image. The 
Portuesi reference discloses the encoded information defines a uniform network resource locator 
embedded in the video (Portuesi 3:15-16). 

As to claim 41, the claimed further comprising the step of receiving over the network 
information relating to the keyword. The Portuesi reference discloses if a URL is activated, 
URL processing unit 72 can invoke Web browser 76 to retrieve the resource located at the 
activated URL and display the resource to the user (Portuesi 9:32-35). 

As to claim 43, the claimed wherein the keyword is embedded in the video image. The 
Portuesi reference discloses the encoded information defines a uniform network resource locator 
embedded in the video (Portuesi 3:15-16). 

As to claim 44, the claimed fiirther comprising the step of searching a network for 
information relating to the keyword. The Portuesi reference discloses if a URL is activated, 
URL processing unit 72 can invoke Web browser 76 to retrieve the resource located at the 
activated URL and display the resource to the user (Portuesi 9:32-35) where accessing a URL 
requires searching at least in regard to a look up table for finding the location of the required 
resource on the Internet. 

As to claim 45, the claimed receiving content comprising a video image; receiving a 
keyword associated with the video image; receiving a code relating to the keyword; and 
receiving information relating to the keyword and the code. The Portuesi reference discloses in 
the embodiment of FIG. 5, decoding system 56 includes a video receiver 66 which receives the 
video signal form distribution network 54. Video receiver 66 then provides the video signal to 
URL decode unit 68. URL decode unit 68 is operable to strip out the URL data firom the 
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encoded video signal (Portuesi 9:19-24). The Portuesi reference also discloses as used herein, the 
term "uniform network resource locator" denotes an identifier of a network document or other 
resource, formatted in accordance with a uniform network protocol, such that computer clients of 
the network for example, can request the document or resource from computer servers of the 
network by reference to the identifier. Intemet/intranet URLS, formatted in accordance with 
HTTP protocol, are one example of uniform network resource locator as that term is used herein 
(Portuesi 4: 16-24), where the examiner views the "http://" of the URL to be a code to facilitate 
access of related information by indicating access using the HTTP protocol. The Portuesi 
reference discloses if a URL is activated, URL processing unit 72 can invoke Web browser 76 to 
retrieve the resource located at the activated URL and display the resource to the user (Portuesi 
9:32-35), 

As to claim 46, the claimed further comprising the step of searching a network for 
information relating to the keyword and the code. The Portuesi reference discloses if a URL is 
activated, URL processing unit 72 can invoke Web browser 76 to retrieve the resource located at 
the activated URL and display the resource to the user (Portuesi 9:32-35) where accessing a URL 
requires searching at least in regard to a look up table for finding the location of the required 
resource on the Internet. 

Claim Rejections - 35 USC§103 
6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
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having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

7. Claim 25 is rejected under 35 U.S.C. 103(a) as being unpatentable over Brodsky (US 
5,809,471) in further view of Farber et al. (US 5,819,284). 

As to claim 25, the claimed wherein the received information relating to the keyword is 
based up a user profile. The Brodsky reference discloses upon recognition of the request, a 
search is initiated to access, import and deliver to the user the information required to satisfy the 
request (Brodsky 3:64-66). However, the Brodsky reference does not disclose the user of a user 
profile. The Farber et al. reference discloses user profile database 174 contains information for 
each user of the system, specifying (a) the categories or types of information services that are to 
be provided to that user, and (b) for those information services, the parameters that are associated 
with the desired information. For example, a first user may desire traffic, financial and sports 
information, a second user may desire weather and news information, and a third user may desire 
traffic, news and weather. For each of these three users, the detailed information desired may be 
different. Thus, the first user may desire traffic information for certain roadways, financial 
information for certain securities, and sports information for particular teams (Farber et al. 4:43- 
55). Therefore, the examiner submits that it would have been obvious to one of ordinary skill in 
the art at the time the invention was made to modify the Brodsky keyword search with the Farber 
et al. profile database for the purpose of providing targeted information to the user that are 
directed towards his/her preferences. 

8. Claims 24, 29-35, 37-41 , 43, and 44 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Brodsky (US 5,809,471) in further view of Portuesi (US 6,499,057). 
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As to claim 24, the claimed further comprising the step of displaying the keyword 
embedded in the video image in response to a selection of the video image. The Brodsky 
reference discloses the user interface 1 10 is a means for the user receiving the radio, TV or 
telephone signals to make requests by voice or other input request means for more video or audio 
information about a topic recently seen or heard. The user interface gives a viewer or listener the 
ability to directly or remotely make requests to an application and to view the response. Options 
are provided for the user to view or hear the 'words' that are valid requests. Options may be 
provided for other forms of requests, say by button selection from a visual or audio menu of 
valid requests (Brodsky 5:1 1-20). However, the Brodsky reference does not disclose displaying 
the keyword embedded in the video image in response to a selection of the video image. The 
Portuesi reference discloses display window 28 can include a caption 34 which provides a 
description of the area within display window 28 over which a pointing device, such as a mouse 
pointer, is positioned. For example, if the pointing device is positioned over hypertext link 22, 
caption 34 can provide a name for link 32 or provide the actual URL (Portuesi 6:22-27). 
Therefore, the examiner submits that it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to modify the Portuesi method of viewing keywords with 
the Portuesi display of keywords when a portion of an image is selected for the purpose of 
making keywords readily available to the user in addition to providing a more intuitive method 
of indicating the keyword for an associated item of interest. 

As to claim 29, the claimed further comprising the step of selecting the video image. See 
rejection of claim 24. 
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As to claim 30, the clamed further comprising the step of displaying the keyword in 
response to the selecting of the video image. See rejection of claim 24. 

As to claim 31, the claimed displaying a video image; selecting at least a portion of the 
video image; and displaying a keyw^ord associated with the portion of the video image in 
response to the selecting of the portion of the video image. The Brodsky reference discloses the 
program may be received via any transmission means including radio, TV and telephone. The 
apparatus includes a recognition system which can recognize the user's request and cause its 
satisfaction to be provided. This is accomplished by creating a dynamically changing dictionary 
of items or keywords extracted from the most recently received program portion (Brodsky 3:56- 
62). The Brodsky reference also discloses options are provided for the user to view or hear the 
'words' that are valid requests. Options may be provided for other forms of requests, say by 
button selection from a visual or audio menu of valid requests (Brodsky 5: 1 7-20). However, the 
Brodsky reference does not disclose selecting a portion of a video image and displaying a 
keyword associated with the portion of the video image in response to the selecting of the portion 
of the video image. The Portuesi reference discloses display window 28 can include a caption 34 
which provides a description of the area within display window 28 over which a pointing device, 
such as a mouse pointer, is positioned. For example, if the pointing device is positioned over 
hypertext link 22, caption 34 can provide a name for link 32 or provide the actual URL (Portuesi 
6:22-27). Therefore, the examiner submits that it would have been obvious to one of ordinary 
skill in the art at the time the invention was made to modify the Brodsky keyword display with 
the Portuesi display window and caption for the purpose of providing a method of making 
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keywords more readily available to the viewer in addition to a more intuitive method of 
identifying keywords associated with a desired object. 

As to claim 32, the claimed further comprising the step of sending the keyword over a 
network. The Brodsky reference discloses upon recognition of the request, a search is initiated 
to access, import and deliver to the user the information required to satisfy the request (Brodsky 
3:64-66). The Brodsky reference also discloses a service provided by the movie program 
producer, broadcaster or a dial-up service provider, gives program relevant information upon a 
user's request (Brodsky 6:54-56). The Brodsky reference also discloses the viewer chooses the 
specific desired topic from the displayed menu. For example, if the user input the name of a 
character for whom additional information is desired, the name is compared to the menu 
dictionary. The recognition processor then compares the name against the valid names in the 
displayed menu. Upon making the match, the processor transmits the match to the database 
application (Brodsky 6:60-67). 

As to claim 33, the claimed further comprising the step of receiving over the network 
information relating to the keyword. The Brodsky reference discloses upon recognition of the 
request, a search is initiated to access, import and deliver to the user the information required to 
satisfy the request (Brodsky 3:64-66). The Brodsky reference also discloses a service provided 
by the movie program producer, broadcaster or a dial-up service provider, gives program 
relevant information upon a user's request (Brodsky 6:54-56). The Brodsky reference also 
discloses the viewer chooses the specific desired topic firom the displayed menu. For example, if 
the user input the name of a character for whom additional information is desired, the name is 
compared to the menu dictionary. The recognition processor then compares the name against the 
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valid names in the displayed menu. Upon making the match, the processor transmits the match 
to the database application. The desired information is retrieved and displayed as a subtitle type 
text, while the viewer is still watching the movie (Brodsky 6:60-67; 7:1-2). 

As to claim 34, the claimed further comprising the step of searching a network for 
information relating to the keyword. The Brodsky reference discloses upon recognition of the 
request, a search is initiated to access, import and deliver to the user the information required to 
satisfy the request (Brodsky 3:64-66). The Brodsky reference also discloses a service provided 
by the movie program producer, broadcaster or a dial-up service provider, gives program 
relevant information upon a user's request (Brodsky 6:54-56). 

As to claim 35, the claimed wherein the keyword is embedded in the video image. The 
Brodsky reference discloses the context extractor 102 has a data processor to extract words 
and/or items from an audio, video or telephony signal (Brodsky 4:40-42) where embedded 
keywords are inherent in order for extraction to occur. 

As to claim 37, the claimed displaying a video image; selecting at least a portion of the 
video image; and sending over a network a keyword associated vAth the portion of the video 
image. The Brodsky reference discloses The context extractor 102 has a data processor to 
extract words and/or items from an audio, video or telephony signal. The extractor 102 samples 
the incoming signal and identifies 'words' and transmit those 'words' to the buffer for storage in 
the dictionary. The selection criteria may include matching all acceptable 'words' against an 
application-supplied listing of available topics. The signal is also passed undisturbed onto its 
normal display 108, say a TV set, radio or a telephone (Brodsky 4:40-48). The Brodsky 
reference also discloses upon recognition of the request, a search is initiated to access, import 
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and deliver to the user the information required to satisfy the request (Brodsky 3 :64-66). The 
Brodsky reference also discloses a service provided by the movie program producer, broadcaster 
or a dial-up service provider, gives program relevant information upon a user's request (Brodsky 
6:45-48). However, the Brodsky reference does not disclose selecting at least a portion of the 
video image. The Portuesi reference discloses as shown, when a pointer 38 of a pointing device, 
such as a mouse, is positioned over a hot spot 40, hot spot 40 can be highlighted to indicate to the 
user that pointer 38 is positioned over hot spot 40. Also, caption 34 can display a URL 42 
associated with hot spot 40 (Portuesi 6:36-40). The Portuesi reference also discloses if a URL is 
activated, URL processing unit 72 can invoke Web browser 76 to retrieve the resource located at 
the activated URL and display the resource to the user (Portuesi 9:32-35). Therefore, the 
examiner submits that it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to modify the Brodsky keyword display wdth the Portuesi selectable hot 
spots for the purpose of providing a method of making keywords more readily available to the 
viewer in addition to a more intuitive method of performing searches using keywords associated 
with a desired object. 

As to claim 38, the claimed wherein the keyword includes a code. The Brodsky 
reference discloses it may be desirable to allow the setting of priorities for all 'items' and/or 
'words'. Priorities may be set so that certain application relevant 'items' or 'words' are kept a 
longer than ordinary duration, or even for the whole length of the program being watched 
(Brodsky 5:48-52). The Brodsky reference also discloses preprocessing of key words 
representing potential user requests, decreases system response time when such a request is 
subsequently made. Preprocessing is desirably performed in conjunction with the priorities of 



Application/Control Number: 09/489,596 Page 21 

Art Unit: 2614 

particular items or words. It may only search for items or words assigned with at least a 
moderately high priority level, and actually retrieve information only for items or words assigned 
with a higher priority level than that of the moderately high priority level (Brodsky 6:3-1 1). 

As to claim 39, the claimed further comprising the step of displaying the keyword 
associated with the portion of the video image in response to the selecting of the portion of the 
video image. The Brodsky reference discloses options are provided for the user to view or hear 
the 'words' that are valid requests. Options may be provided for other forms of requests, say by 
button selection from a visual or audio menu of valid requests (Brodsky 5:17-20). However, the 
Brodsky reference does not disclose selecting a portion of a video image and displaying a 
keyword associated with the portion of the video image in response to the selecting of the portion 
of the video image. The Portuesi reference discloses display window 28 can include a caption 34 
which provides a description of the area within display window 28 over which a pointing device, 
such as a mouse pointer, is positioned. For example, if the pointing device is positioned over 
hypertext link 22, caption 34 can provide a name for link 32 or provide the actual URL (Portuesi 
6:22-27). Therefore, the examiner submits that it would have been obvious to one of ordinary 
skill in the art at the time the invention was made to modify the Brodsky keyword display with 
the Portuesi display window and caption for the purpose of providing a method of making 
keywords more readily available to the viewer in addition to a more intuitive method of 
identifying keywords associated with a desired object. 

As to claim 40, the claimed wherein the keyword is embedded in the video image. The 
Brodsky reference discloses the context extractor 102 has a data processor to extract words 
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and/or items from an audio, video or telephony signal (Brodsky 4:40-42) where embedded 
keywords are inherent in order for extraction to occur. 

As to claim 41, the claimed further comprising the step of receiving over the network 
information relating to the keyword. The Brodsky reference discloses upon recognition of the 
request, a search is initiated to access, import and deliver to the user the information required to 
satisfy the request (Brodsky 3:64-66). The Brodsky reference also discloses a service provided 
by the movie program producer, broadcaster or a dial-up service provider, gives program 
relevant information upon a user's request (Brodsky 6:54-56). The Brodsky reference also 
discloses the viewer chooses the specific desired topic from the displayed menu. For example, if 
the user input the name of a character for whom additional information is desired, the name is 
compared to the menu dictionary. The recognition processor then compares the name against the 
valid names in the displayed menu. Upon making the match, the processor transmits the match 
to the database application. The desired information is retrieved and displayed as a subtitle type 
text, while the viewer is still watching the movie (Brodsky 6:60-67; 7:1-2). 

As to claim 43, the claimed wherein the keyword is embedded in the video image. The 
Brodsky reference discloses the context extractor 102 has a data processor to extract words 
and/or items from an audio, video or telephony signal (Brodsky 4:40-42) where embedded 
keywords are inherent in order for extraction to occur. 

As to claim 44, the claimed further comprising the step of searching a network for 
information relating to the keyword. The Brodsky reference discloses upon recognition of the 
request, a search is initiated to access, import and deliver to the user the information required to 
satisfy the request (Brodsky 3:64-66). The Brodsky reference also discloses a service provided 
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by the movie program producer, broadcaster or a dial-up service provider, gives program 
relevant information upon a user's request (Brodsky 6:54-56). 

9. Claims 36 and 42 are rejected imder 35 U.S.C. 103(a) as being unpatentable over 
Brodsky (US 5,809,471) in further view of Portuesi (US 6,499,057) and Farber et al. (US 
5,819,284). 

As to claim 36, the claimed sending the keyw^ord over a network; and receiving over the 
network information relating to the keyword; wherein the information relating to the keyword is 
based upon a user profile. The Brodsky reference discloses upon recognition of the request, a 
search is initiated to access, import and deliver to the user the information required to satisfy the 
request (Brodsky 3:64-66). The Brodsky reference also discloses a service provided by the 
movie program producer, broadcaster or a dial-up service provider, gives program relevant 
information upon a user's request. However, the Brodsky reference does not disclose the user of 
a user profile. The Farber et al. reference discloses user profile database 174 contains 
information for each user of the system, specifying (a) the categories or types of information 
services that are to be provided to that user, and (b) for those information services, the 
parameters that are associated with the desired information. For example, a first user may desire 
traffic, financial and sports information, a second user may desire weather and news information, 
and a third user may desire traffic, news and weather. For each of these three users, the detailed 
information desired may be different. Thus, the first user may desire traffic information for 
certain roadways, financial information for certain securities, and sports information for 
particular teams (Farber et al. 4:43-55). Therefore, the examiner submits that it would have been 
obvious to one of ordinary skill in the art at the time the invention was made to modify the 
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Brodsky keyword search with the Portuesi hot spots and the Farber et al. profile database for the 
purpose of providing targeted information to the user that are directed towards his/her 
preferences. 

As to claim 42, the claimed wherein the information relating to the keyword is based 
upon a user profile. The Brodsky reference discloses upon recognition of the request, a search is 
initiated to access, import and deliver to the user the information required to satisfy the request 
(Brodsky 3:64-66). However, the Brodsky reference does not disclose the user of a user profile. 
The Farber et al. reference discloses user profile database 174 contains information for each user 
of the system, specifying (a) the categories or types of information services that are to be 
provided to that user, and (b) for those information services, the parameters that are associated 
with the desired information. For example, a first user may desire traffic, financial and sports 
information, a second user may desire weather and news information, and a third user may desire 
traffic, news and weather. For each of these three users, the detailed information desired may be 
different. Thus, the first user may desire traffic information for certain roadways, financial 
information for certain securities, and sports information for particular teams (Farber et al. 4:43- 
55). Therefore, the examiner submits that it would have been obvious to one of ordinary skill in 
the art at the time the invention was made to modify the Brodsky keyword search with the 
Portuesi hot spots and the Farber et al. profile database for the purpose of providing targeted 
information to the user that are directed towards his/her preferences. 

10. Claims 25, 36, 42, and 47 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Portuesi. 
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As to claim 25, the claimed wherein the received information relating to the keyword is 
based upon a user profile. The Portuesi reference discloses if a URL is activated, URL 
processing unit 72 can invoke Web browser 76 to retrieve the resource located at the activated 
URL and display the resource to the user. However, the Portuesi reference does not specifically 
disclose related information based on a user profile. Nevertheless, the examiner gives Official 
Notice that it is notoriously well known in the art of accessing web pages, to inhibit the display 
of certain web pages if the content is undesirable to the user, such as adult content, and to block 
such content according to a user profile. Therefore the examiner submits that it would have been 
obvious to one of ordinary skill in the art at the time the invention was made to modify the 
Portuesi URL access accordingly for the stated advantages. 

As to claim 36, the claimed sending the keyword over a network; and receiving over the 
network information relating to the keyword; wherein the information relating to the keyword is 
based upon a user profile. The Portuesi reference discloses if a URL is activated, URL 
processing unit 72 can invoke Web browser 76 to retrieve the resource located at the activated 
URL and display the resource to the user (Portuesi 9:32-35). The Portuesi reference also 
discloses this allows a user to activate links and connect to resources, for example, across the 
public Internet or private intranets during playback of the time-based medivrai. The Portuesi 
reference also discloses if a URL is activated, URL processing unit 72 can invoke Web browser 
76 to retrieve the resource located at the activated URL and display the resource to the user 
(Portuesi 9:32-35). However, the Portuesi reference does not specifically disclose related 
information based on a user profile. Nevertheless, the examiner gives Official Notice that it is 
notoriously well known in the art of accessing web pages, to inhibit the display of certain web 
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pages if the content is undesirable to the user, such as adult content, and to block such content 
according to a user profile. Therefore the examiner submits that it would have been obvious to 
one of ordinary skill in the art at the time the invention was made to modify the Portuesi URL 
access accordingly for the stated advantages. 

As to claim 42, the claimed wherein the information relating to the keyword is based 
upon a user profile. The Portuesi reference discloses if a URL is activated, URL processing unit 
72 can invoke Web browser 76 to retrieve the resource located at the activated URL and display 
the resource to the user. However, the Portuesi reference does not specifically disclose related 
information based on a user profile. Nevertheless, the examiner gives Official Notice that it is 
notoriously well known in the art of accessing web pages, to inhibit the display of certain web 
pages if the content is undesirable to the user, such as adult content, and to block such content 
according to a user profile. Therefore the examiner submits that it would have been obvious to 
one of ordinary skill in the art at the time the invention was made to modify the Portuesi URL 
access accordingly for the stated advantages. 

As to claim 47, the claimed wherein the code comprises a numerical tag. The Portuesi 
reference discloses as used herein, the term "uniform network resource locator" denotes an 
identifier of a network document or other resource, formatted in accordance v^th a uniform 
network protocol, such that computer clients of the network for example, can request the 
document or resource from computer servers of the network by reference to the identifier. 
Internet/intranet URLS, formatted in accordance with HTTP protocol, are one example of 
uniform network resource locator as that term is used herein (Portuesi 4: 16-24), where the 
examiner views the "http://" of the URL to be a code to facilitate access of related information 
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by indicating access using the HTTP protocol. However, the Portuesi reference does not 
specifically disclose the use of a code comprising a numerical tag. Nevertheless, the examiner 
gives Official Notice that it is notoriously well known in the art to use numerical tags to indicate 
specific items within a website such as the use of product id numbers for direct access to a 
particular product on an online shopping website. Therefore, the examiner submits that it would 
have been obvious to one of ordinary skill in the art at the time the invention was made to 
modify the Portuesi HTTP code to include a product ID number for the purpose of facilitating 
direct access to a product on a designated website. 

Conclusion 

1 1 . Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS fi-om the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated fi-om the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS firom the date of this 
final action. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Johnny Ma whose telephone number is (703) 305-8099. The 
examiner can normally be reached on 8:00 am - 6:00 pm (First Friday off). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Miller can be reached on (703) 305-4795. The fax phone numbers for the 
organization where this application or proceeding is assigned are (703) 308-5399 for regular 
communications and (703) 308-5399 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 305-3900. 
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